Rethinking AI benchmarks: A new paper challenges the status quo of evaluating artificial intelligence

Benchmarks like the bar exam are usually good measures of human competence, but can be misleading when used to evaluate AI systems.